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In the classical quickest detection problem, one must detect as 
quickly as possible when a Brownian motion without drift “changes” 
into a Brownian motion with positive drift. The change occurs at an 
unknown “disorder” time with exponential distribution. There is a 
penalty for declaring too early that the change has occurred, and a 
cost for late detection proportional to the time between occurrence 
of the change and the time when the change is declared. Here, we 
consider the case where there is also a cost for observing the process. 

This stochastic control problem can be formulated using either the 
notion of strong solution or of weak solution of the s.d.e. that defines 
the observation process. We show that the value function is the same 
in both cases, even though no optimal strategy exists in the strong 
formulation. We determine the optimal strategy in the weak formula¬ 
tion and show, using a form of the “principle of smooth fit” and under 
natural hypotheses on the parameters of the problem, that the opti¬ 
mal strategy takes the form of a two-threshold policy: observe only 
when the posterior probability that the change has already occurred, 
given the observations, is larger than a threshold A > 0, and declare 
that the disorder time has occurred when this posterior probability 
exceeds a threshold B > A. The constants A and B are determined 
explicitly from the parameters of the problem. 

1. Introduction. The classical quickest detection problem [23], Chap¬ 
ter 4.4, is as follows. One observes a stochastic process X = {Xt)t>o that 
solves the stochastic differential equation (s.d.e.) 

( 1 . 1 ) dXt = rl^9<tydt + adWt. 
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Here, r > 0, a > 0, W = {Wt)t>Q is a standard Brownian motion, and 9 
is a nonnegative random variable that is independent of (VHt), sometimes 
called a “disorder time,” or a “change point.” The random variable 0 is not 
observed directly, but only through its effect on the sample paths of X. When 
t <6, the observer is simply watching a Brownian motion, but when t>9, 
a drift (or signal) with intensity r appears. The observer seeks to detect as 
quickly as possible the appearance of this signal, while keeping sufficiently 
low the probability of a “false alarm,” that is, declaring that the signal 
has appeared when, in fact, it has not. Typically, the distribution of 0 is 
assumed known, and, given 0 > 0, even equal to an exponential distribution 
with known parameter A > 0; see [19] for many variations on this problem 
and for numerous references. 

In this paper, we consider the situation where there is an observation cost 
6 > 0 per unit time, and the observer can choose to observe or not. When 
he does not observe, the process X is constant {dXt = 0), and when he does 
observe, X satisfies (1.1). The objective is to detect the appearance of the 
signal as quickly as possible, while keeping low the probability of false alarm 
and the cost of observation. Therefore, the problem is no longer an optimal 
stopping problem but an optimal stopping/control problem. 

There are several papers in the literature that consider this type of prob¬ 
lem, in which there is either a cost or a constraint on observations. A dis¬ 
cussion already appears in Bather [5], with a precise continuous-time for¬ 
mulation given in Balmer [1, 2]: he allows only a restricted class of policies 
and uses a different cost function than the one we define in (2.4) below. 
Dayanik [9] considers a continuous-time problem with observations allowed 
only at fixed times. Banerjee and Veeravalli [3, 4] consider a discrete-time 
formulation, in which observations are costly only if they occur before the 
alarm time. They show that a two-threshold policy is asymptotically opti¬ 
mal. Finally, Bayraktar and Kravitz [6] consider a continuous-time problem 
in which observations are allowed only at a discrete set of times that is 
determined adaptively. 

In this paper, we consider that the control h = {ht)t>o is a [0, l]-valued 
process, where ht = 1 means that observation occurs, and ht = 0 means 
absence of observation. Therefore, the observation process is described by 
the stochastic differential 

(1.2) dXt = rhtl{B<t}dt +a^/JhdWt, A'o = 0. 

Note that when ht G {0,1}, the square-root has no effect. However, it will be 
convenient during the resolution of the problem to consider also ht G [0,1], 
and since we are free to decide the formulation when 0 < ht < 1, we have 
chosen to use (1.2). 

We assume that all objects are defined on a filtered probability space 
(HjT', (Tt)t>o,H). Therefore, Wt = Wt{io), 9 = 9{uj), and ht = ht{oj). The 
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assumption that ht depends on a; {ht = ht{ijj)) does not create difficulties 
with definition of the (“/i-controlled”) process X via formula (1.2). However, 
we must define precisely what information the observer can use to decide to 
switch from one value of ht{co) to another. 

It is reasonable to assume that the control function ht depends on u via 
the observation process: ht{uj) = ht{X{uj)). In this case, the s.d.e. (1.2) will 
take the form 

(1.3) dXt = rht{X)l^g<t} dt + (X) dWt, 

and, inevitably, we have to explain how to formulate this s.d.e. and give a 
precise definition of the control h = {ht{X))t>o- 

These questions are considered in Section 2, where we give two precise but 
distinct formulations of the notion of a solution of equation (1.3), according 
to whether we interpret X as a strong or weak solution of (1.3). Then we 
derive some preliminary properties of the sufficient statistic vr^, which is 
the conditional probability, given the observations (X^, s E [0,t]), that 9 <t. 
In Section 3, we study the law of vr^, writing it, and the likelihood ratio 
ip^ = TTt /{I — TT^), as solutions of diffusion equations in the filtration of 
the observed process. In this section, we also establish, in the spirit of [11] 
and [18], a “verification lemma” (Lemma 3.7) that gives sufficient conditions 
for the optimality of a strategy. 

In Section 4, we give the form of a candidate optimal strategy and associ¬ 
ated candidate value function and derive the ordinary differential equations 
with two free boundaries that characterize this function. These are com¬ 
pleted by imposing boundary conditions that imply continuity and an ap¬ 
propriate degree of smoothness at the boundaries; see (4.10)-(4.14). These 
equations are then solved completely, up to the resolution of a transcenden¬ 
tal equation; see (4.26). The form of the solution depends on the value of 
the observation cost b, and it turns out that there are three regimes: if b is 
large enough, then it is best never to observe, and to stop simply when the 
posterior probability exceeds a certain threshold B £]0,1[. For smaller 
positive values of b, there are two thresholds < A < B <1 such that it is 
best not to observe when < A, to observe when vr^ G ]^) I^[ and to declare 
an alarm when > B. The candidate value function is given in Proposi¬ 
tions 4.4 and 4.5, depending on the size of b. The third regime is when 6 = 0, 
which is the classical case of [23] and corresponds to 0 = A < B < 1. 

For small positive values of 6, the candidate value function and optimal 
strategies are such that it is not clear whether an optimal strategy does 
indeed exist! In fact, in the strong formulation, no optimal strategy exists in 
general, but such an optimal strategy does exist in the weak formulation. It 
turns out, however, that the value function is the same in both formulations. 
We discuss this question at the end of Section 4. 
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In Section 5, we show that the candidate value function of Section 4 is 
indeed the value function in both the weak and strong formulations (Theo¬ 
rems 5.1 and 5.2). However, because of the absence of an optimal strategy 
in the strong formulation, it is not possible to conclude directly from a “ver¬ 
ification lemma” (Lemma 3.7) that the candidate value function is indeed 
the value function in the strong formulation. Therefore, we use a different 
approach in Theorem 5.2; for e > 0, we consider strategies that approximate 
the candidate optimal strategy but are defined via s.d.e.’s with sufficiently 
smooth coefficients. We then compute explicitly the cost associated with 
these strategies. This requires computing the expected time to hit a thresh¬ 
old, which, in turn, requires solving another o.d.e. [given in (5.11)]. We do 
this in Section 5, and in Proposition 5.7, we show by direct calculation that 
the expected costs of the approximately optimal strategies converge to the 
candidate value function, proving that this is indeed the value function in 
the strong formulation. 

2. Stating the problem. Consider a filtered probability space (H,J^, 
{J^t)t>o,P) with a filtration (T)) (satisfying the usual hypotheses [21]). Let 
0 be a random variable defined on H that is -measurable. We assume that 
there are tto G [0,1] and A > 0 such that 

(2.1) p{e = 0} = TTo and P{e > x \ B > 0} = e-^^. 

We let W = {Wt)t>o be a standard Brownian motion adapted to {Pt)t>o 
such that for all t>0, the process {Wg+t — Wt,s>0) is independent of Tf 
In particular, {Wt)t>o is independent of 6. 

Controls and stopping times. 

Definition 2.1. A progressively measurable process h = {ht{uj))t>o de¬ 
fined on with values in [0,1] will be called a stochastic 

control. 

Let C'(M+,M) denote the space of continuous functions from M+ to M. 

Definition 2.2. A canonical control h = {ht{x))t>o is a map {t,x)>-^ 
ht{x) from M+ X (7(1^+,^) to [0,1] that is progressively measurable for the 
canonical hltration on (^'(M+jM). 

A canonical stopping time t = t{x) is a random variable t:C'(M+,M) —)• 
M+ that is a stopping time relative to the canonical filtration on C (M+, M). 

Definition 2.3. A stochastic control h = (/it(w))t>o is called an ad¬ 
missible control if it has the form ht{u) = ht{X{u)) for a canonical control 
ht{x), and the s.d.e. 

(2.2) dXt = rht{X)l^g<t} dt + a^ht{X) dWt, 


Xq — 0 
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admits a strong solution in the sense of the next definition (Definition 2.4). 


Definition 2.4. Assume that a filtered probability space 
{J^t)t>o,P) is given a priori together with a random variable 9 = 6{uj) which 
is J^o-measurable and satisfies (2.1), and with a Brownian motion W(uj) = 
{Wt{u}))t>o such that Wt is J't-measurable, for all t>0. 

A strong solution of the s.d.e. (2.2) is a continuous stochastic process 
X = {Xt{io))t>o that satishes (2.2) and Xt is /"f-measurable, for all t>0. 

One may consider also the case where (2.2) has a weak solution. 


Definition 2.5. We assume that a canonical control h = {ht{x))t>o and 
the law of 9 in (2.1) are given a priori. A weak solution of the s.d.e. (2.2) is 
a system of the following objects: 


- a filtered probability space (D, X, {Xt)t>o,P) (which is not given a priori); 

- a Brownian motion W = {Wt)t>Q such that Wt is J^^-measurable, for all 

t >0; 

- an J^Q-iaeasurable random variable 9 with the law specified in (2.1); 

- an (J))t>o-adapted process X = {Xt)t>o which satisfies the s.d.e. (2.2), 
that is, for all t > 0 


(2.3) 


Xt = 


f rhs{X)l^g^s}ds + [ a^hs{X) dWg 
Jo Jo 


Definition 2.6. For the case of strong solutions, a strategy is a pair 
{h,T), where h = {ht{X{uj)))t>o, T = r(A(a;)) for some canonical control 
{ht{x))t>o and canonical stopping time t{x). 

For the case of weak solutions, {h,T,X) is called a control system. 


Cost. 


Definition 2.7. The cost associated with a strategy (/i,r) or a control 
system {h,T,X) is 


(2.4) 


C{h,T) = C{h,T,X) 


r{X) 


— ^{T(x)<e} + a(T(^) — ^)1 {t(x)>0} + ^ / ht{X) dt, 

Jo 


where a > 0, so as to penalize late detection of the alarm time 0, and 6 > 0. 


Since the case 6 = 0 is covered in [23], Chapter 4.4, we will focus on the 
case 6 > 0. 
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Objective. Our first objective is to find the value 

V= inf E(C(h,T)), 

(h,T) 

where the infimum is over all strategies, and to hud an optimal strategy 
{h*,T*) that achieves this inhmum, or at least, to find a strategy that is 
within £ of this infimum (e > 0). A second objective is to find the value 

inf E{C{h,T,X)), 

{h.fT,X) 

where the infimum is over all control systems, and an optimal control system 
{h*,T*,X*). Clearly, < C. 

Dependence on ttq. The quantities V and C" are in fact functions of the 
number ttq = P{9 = 0}, which we denote ^(tto) and ^’"(tto): 

(2.5) g{7ro)= inf E{C{h,T)), 

(h,T) 

(2.6) 5'^(7ro)= inf £;(C'(/i,t,X)). 

{h,T,X) 

Clearly, g'^ < g. The following simple lemma (see also [20], Section 2.7) 
provides important information about the form of these two functions. 


Lemma 2.8. The functions g and g^ are concave. 


Proof. By the law of total probability. 


E{C{h,T)) 

f MX) 

= 7roEUT{X) + b ht{X) 


dt 


= 0 


+ (1 - xo) 

f MX) 

X 1{^(X)<0} + a(r(X) — 0)l{7-(x)>0} + 5 J ht{X) 


dt 


0>O 


We note that the first expectation does not depend on vro, since t{X) and 
ht{X) are determined by the observation process only, and the second does 
not either, since the conditional distribution of 0 given that 0 > 0 does not 
depend on tto. Therefore, tto e-)• E{C{h,T)) is an affine function of tto, and g, 
being the inhmum of affine functions, is concave. The same argument applies 
to . □ 
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Sufficient statistic. Let J-^ = {J'^) be the natural filtration of the ob¬ 
served process X, augmented with P-null sets. Let (vr^) be the optional 
projection of (l{6Kt},t > 0) onto this filtration, so that for all t, = P{6 < 
t I < t} a.s. The next several lemmas are identical both for strategies 
and for control systems, so we state them only for strategies. 


Lemma 2.9. With the above notation, 

(2.7) E{C{h,T)) = e( 1 — TT^ + a f Tr^ds + b f hgds 

V Jo Jo y 

Proof. Note that i?(l{.r<6»}) = -^'(1 “ '^t) 

( POO \ POO 

'^{e<s}'^{s<T}ds] = £’(7r^l{^<^})ds 


= E 

This proves the lemma. □ 


TTgds 


According to Lemma 2.9, the expected cost associated to a strategy (h,r) 
is the expectation of an adapted functional of the posterior probability pro¬ 
cess {tti). Therefore, it will be natural to express controls as functionals of 
(vr^). We proceed with the analysis of this process. 

3. Semimartingale characteristics of (tt^) and a verification lemma. For 

0 < u < t, let be the conditional distribution, given that 6 = u, of X 
restricted to [0, t], and let /ij be the unconditional distribution of X restricted 
to [0,t]. 

Lemma 3.1. The Radon-Nikodym derivative of p,u^t with respect to p,t,t 
is 

Proof. Recall Girsanov’s theorem [17], Theorem 8.6.6, page 166: let 

dZt = a{Zt)dWt, 
dZt = 'ytdt + a{Zt) dWt, 

and suppose that under P, the process (Wt) is a standard Brownian motion. 
Define P by 
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If Ep{^) = 1, then the law of (Zt) under P is the same as the law of {Zt) 
under P. 

If 0 = rt, then the law of (X*, s < t) is the same as that of (1^, s < t), where 
(3.2) dYs = rhs{Y)l{u<s} ds + ay/hs{Y) dWg, 0<s<t. 

If 6 = t, then the law of (Xs,s < t) is the same as that of {Zs,s < t), where 
dZg = a\/hs{Z) dWs, 0 < s < t. 

Therefore, for A G 5(C([0,t],M)), 

IXuM) = P{y- e ^} = Ep{Ia{Y.)) = Ep , 

where P is defined by 



Note in particular that Novikov’s condition [17] is satisfied. Using (3.2), we 
see that this can be written 

Therefore, by Girsanov’s theorem, 

Pu,t{A) = Ep(^A{Z.)(xp(^j -^dZs-^J hs{Z)ds^'^ 

This proves Lemma 3.1. □ 


Let Fe denote the probability distribution function of 0, so that 
r 0, if X < 0, 

l7ro + (l-7ro)(l-e“^’’’), if x > 0. 


Lemma 3.2. 


(note that the 0— 


We have 



accounts for the discontinuity of Fq at Oj. 
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Proof. The notation now refers to the right-hand side of (3.1), 
which is continuous in u. For the first equality in the lemma, it suffices to 
show that for all B G ;S(C'([0,f],M)), 


E 1 




/o_ diR 

To see this, observe that 

I dP{uj) f ^^^l^{ij)Fe{du) = j Fg{du) f 

d{^l[ 0 .t] 6 B} Jo- dut Jo- J{x\io,t]m dut 


— / FQ{du)iiu,t{J^\[o,t]^ F} 

Jo- 

= P{d<f,X|[o,t]Gi?}. 

This proves the first equality. The second is a consequence of the chain rule 
for Radon-Nikodym derivatives. □ 


Lemma 3.3. We have 

dut, 


1 

l-TTj = 


d^-H 


^ ' Fg{du) = {1 - Tro)e 

dfit 


i: 


Proof. As in Lemma 3.2, one checks that 


r 

p{e>t\Xs,s<t} = j 


dfit 


Fe{du). 


Since = 1 when u> t, the right-hand side is equal to 

r+oo 


r+oo 

Jt dnt,t dut 


dHu,t d^t,t 7-, / T \ d^t,t 
-^- -Fg{du) = 

dut 


Fg{du). 


This proves the first equality in the statement of the lemma. The sec¬ 
ond equality is a consequence of the fact that for u > 0, Fg{du) = (1 — 
7ro)Ae“'^“ dw. □ 

Set 


= 


vrr 


1 -vr'^ 


rt r 1 

Z'll -f- / a ^ ; 

’ Ju 2 a2 


/lo ds. 


and let 
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Use Lemmas 3.1, 3.2 and 3.3 to see that 

oAt rt 


gAt rt 

iPt=- - / exp{Zu,t)F0{du) 

1 - TTo Jo- 

gAt rt 

(3.3) =--exp(Zo,t) / exp{-Zo^u)F 0 {du) 

1 - TTo Jo- 

gAt / rt 

= -exp(Zo,t) TTo + (1 - TTo) / exp(-Zo,„)Ae 

1 “ ^0 V Jo 



Lemma 3.4. The following s.d.e. is satisfied: 

(3.4) dip’l = X{1 + (f^) dt +dXf. 


Proof. Observe from (1.2) that the quadratic variation of Xt is d{X)t = 
a‘^htdt, so we can apply Ito’s formula and (3.3) to get 


dip^ = \p)^ dt 


^Xt 


+ 


1 - TTo 
t 

X ^ 

'o- 

\t 


exp(Zo,t) -^dXt - 




r'^ht . 1 / 

- "pr dt — I —pr 

2a^ 2 V 0-2 


a'^ht dt 


/ exp{-Zo^u)F0{du) 

Jo- 

+ e''* exp(Zo,t) exp(-Zo,f)Ae“^* dt 


— A(1 + ip^) dt H —yTt dXf. 




□ 


Lemma 3.5. The process X = {Xt)t>o has the stochastic differential 
dXt = rht{X)TTtdt + a\/h'tdWt, 
where {Wt) is a standard Brownian motion. 


Proof. Observe that 

dXt - rhtTTtdt = {rhtl^0<t} - rhtTTt) dt + ay^tdWt, 

and the right-hand side has mean zero (given and quadratic variation 

a'^htdt. Further, the left-hand side is adapted to , so that the right- 
hand side is too, and has mean zero. In particular, it is the differential 
of a local -martingale with quadratic variation ay/hidt. According to 
[15], Chapter 3, Theorem 4.2, this term is equal to ay/ht times a standard 
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Brownian motion increment. We note for future reference that (Wt) need 
not be J^^-adapted, but the martingale 

(3.5) Mt = a [ y/h'sdWt = Xt- f rh,(X)7r,ds 

Jo Jo 

is clearly J^^’^'-adapted. □ 


Lemma 3.6. Setp = —. Then 

• a 

2 

(3.6) = A(1 — vr^) dt H —~ - 2 ~ dt 

and 

(3.7) dvrf = A(1 — vrf) dt + pvr^(l — 'K^)-\/litdWt. 


Proof. Note that 7r(* = + where f{x) = x{l + x)~^. 

Since f'{x) = {l + x)~‘^ and f"{x) = —2(l + x)“^, Ito’s formula and Lemma 3.4 
yield 

<i4 = /'(»>f)<i4 + L"{4)<i(/), 


1 


(1 + 


-^(1 + Tt)dt + dXt ) + - 


1 -2 


2{l + p>^Y\a 


a htdt. 


Recall that 1 + to see that this is equal to 


A(1 - 7rf) dt + ^7r^^(l - vrf) dXt -^(7rf)^(l - 7r^)htdt, 






which establishes (3.6). By Lemma 3.5, this is equal to 


A(1 -'K^)dt + - T^t){'''dtT^t dt + a\/ht dWt) - ^(7r^)j(l -vrf)/it dt, 

which simplifies to 

A(1 - 7rf) dt + - 'K^)\/litdWt. 

a 

This establishes (3.7). □ 


Strategies expressed in terms of (vr^). According to (3.7), (tt^) is a dif¬ 
fusion process, and therefore an optimal canonical control will typically be 
expressed as a function of vr^*; that is, we will mainly be interested in controls 
ht{X) of the form ht{X) = h{t,TT^), where /i:M+ x [0,1] —)• [0,1] is measur¬ 
able and given. We explain here how to describe the observation process and 
the admissible control (/it) associated with such a function h. 
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Consider the s.d.e. 

dpt = A(1 -pt)dt 

(3.8) - Pt){'<'h{t,pt)l{e<t} dt + ay/h{t,pt) dWt) 


- -pt)Kt,pt)dt, 




with pq = P{6 = 0}. Assume that h is such that (3.8) has a strong solution 
[i.e., an (J't)-adapted solution]. Then we define the observation process by 
Aq = 0 and 


(3.9) dXt = rh{t,pt)l{e<t} dt + a^/h{t,pt) dWt- 

This process is adapted to {Ft), and by (3.8), 


dpt = A(1 - pt) dt + -^pt{l - Pt) dXt 


(3.10) 


cr^ 


- ^{pt)'^{l-pt)h{t,pt)dt. 




Let qt = pt/{l — Pt)- Applying Ito’s formula, we hnd that 


(3.11) 


dqt = A(1 + qt) dt + -^qt dXt- 




According to [22], Chapter IX, (2.3), the solution of this linear s.d.e. is 


T 1 

g4 = exp( -^Xt + \t- -^{X)t 

2 


go + 


F f r 1 


Xds 


In particular, qt, and therefore pt, is a function of and we can write 

Pt = ht{X), where {t,x) i-A ht{x) from 1R+ x (^(M+jM) to M is progressively 
measurable. Looking back to (3.9), we see that {Xt) is a strong solution of 
the s.d.e. 


(3.12) 


dXt = rhtiX)l{e<t} dt + ayJht{X)dWt, 


where ht{x) = h{t,ht{x)). Therefore, (ht) is an admissible control. 
Comparing (3.11) and (3.4), we conclude that qt = and therefore 

(3.13) p^ = Tt'i = P{e<t\F^]. 

This means that the control ht{X) is indeed equal to h{t,7r^). 

We note that as in (3.7), there is a Brownian motion (Wt) such that 

(3.14) dpt = X{1 - Pt) dt + ppt{l -pt)\/h{t,pt)dWt. 
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If r is a stopping time defined using vr^, for instance, 

(3.15) r = inf{t > 0: vrf G S'} 
for some Borel set S C [0,1], then 

r = inf{t > 0: ht{X) G Sj, 

so t = t{X) is a canonical stopping time. In particular, {{ht{x)),T{x)) is a 
strategy. 

The above discussion shows that if (3.8) has a strong solution, then we 
can construct a strategy for which (2.1) or (3.12) admits a strong 

solution {Xt), such that pt = vrf, and the expected cost E{C{{ht),T)) is given 
by (2.7). 

In the case where (3.8) admits a weak solution, we would similarly con¬ 
clude that (2.2) or (3.12) admits a weak solution, and considering r as in 

(3.15) , we would conclude that {{ht),T,X) is a control system with the same 
expected cost. 

Verification lemma. For tt G [0,1], let denote expectation in the case 
where vro = vr. Recall that we have defined 

g{TT) = ini E^{C{h,T)), g'^ino) = inf E{C{h,T,X)). 

(h,r) (h^r,X) 

By Lemma 2.8, g is concave, and by Lemma 2.9, 

g{7r) = inf Et^(i — + a [ iTg ds + b f hgds 

V Jo Jo 

with similar properties for g'". According to [11], Theorem 3.67, we ex¬ 
pect to be able to characterize each of these two functions as a function 
g* with certain properties concerning martingales and submartingales. The 
next lemma gives conditions that will allow us to show that a function g* is 
equal to g (resp., g'^) and check that a strategy ((h*),r*) [resp., a control 
system ((/i*),r*,X*)] is optimal. 

Lemma 3.7 (Verification lemma). Suppose that g* is a bounded contin¬ 
uous function defined on [0,1] such that 0< g*{x) <l — x, for all x G [0,1]. 

(1) Suppose that for any vr G [0,1], the following property holds: 

(a) for any strategy {{ht),T) [resp., for any control system {h,T,X)], 
the process (!}) is an -submartingale under Pt,-, where 

(3.16) Yt = g* {tti)-\- a f 7rgds-\-b f hgds. 

Jo Jo 

Then g* <g (resp., g* < g'^J. 
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( 2 ) Suppose that for any vr G [0,1], in addition to (a), the following three 
properties hold: 

(b) for the strategy ((/i*),r*) [resp., the control system , X*)], 

the process is an -martingale under where 

f Tr'f* ds-^bf h^ds; 

Jo Jo 

(c) E^{t*) < +oo; 

(d) 5*(7r^^*‘) = l-7r^^*‘. 

Then g* = g and {{h*),T*) is an optimal strategy [resp., g* = g^, and 
((/i*),r*,X*) is an optimal control system]. 

Proof. We first establish (1). Let {{ht),T) be a strategy. If E{t) = +oo, 
then E{C{h,T)) = +oo. Indeed, by (2.4), E{C{h,T)) > aLl(Tl{.,->e}) — aE{6). 
Since 

E{t) = E{Tl{r>d}) + 

and the second term is no greater than E{6) < +oo, we conclude that 
E^rl^T^gj) = +00 and so E{C{h,T)) = +oo. 

Therefore, in the dehnition of g, we can restrict the inhmum to those 
strategies for which E{t) < +oo. Since 1 — x > g*{x), Lemma 2.9 implies 
that 

E^{C{h,T))>E{Y^). 

Since (Yf) is a submartingale by (a) and t A r is a bounded stopping time, 
ETT{Yt/\r) > Et^{Yq) = g*{iT). By Patou’s lemma in the form Pl(hmsupynAT) > 
hmsup£'(y„AT) (cf. [7], Chapter 1), which applies since E{t) < +oo, we see 
that 

En(Yr) > hmSUp.P,r(yiAr) > g* (t^) ■ 

We conclude that ET^{C{h,T)) > ff*(vr) for all strategies {{ht),T), and there¬ 
fore g > g* ■ The proof for 5 " is identical and is omitted. 

We now establish (2) for g. It suffices to show that g*{TT) = Ej^lfY^.^). 
Indeed, this will complete the proof, since by (d) and Lemma 2.9, 

g*{'K) = E^{Yf,) = E^(^g*{j:'[l) + a 7rfds + bJ^ h*ds 

= E^{Cih*,T*))>g{7T). 

Since we have already proved that g> g*, this shows that g*{7r) = 5 ( 7 r). 

In order to check that fl'*('/r) = ET^iYf*), note that 0 < y/ < 1 -|- (a + b)t 
and Et^{t*) < -|-oo by (c). Therefore, which is a martingale by (b), 

is uniformly integrable. By the optional sampling theorem [10], E(Yf) = 
E{Yq) = g*{Tr). This completes the proof for g. The proof for g'*' is identical 
and is omitted. □ 
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4. A candidate for the value function. We now seek analytical conditions 
on a function g* that will guarantee the properties of Lemma 3.7. Consider 
the process (Yt) defined in (3.16) (we write g instead of g* to simplify the 
notation). By Ito’s formula and Lemma 3.6, 

dYt = g'{T^t) + CLTT^ (It + bht dt 


(4.1) = 


V(Tt'')(l - -Kt) + (1 - + avrf + bht 


dt 


+ 5r'(7rf)-7rf(l - 'K^)^/htdWt. 
a 

Therefore, (Y)) will be a submartingale if the term in brackets is nonnegative, 
for any value of ht ■ Since this term is an affine function of ht , this is equivalent 
to this term being nonnegative for ht = 0 and ht = l, that is, for all x € [0,1], 


(4.2) 


Xg'{x){l — x) + ax > 0 


and 

(4.3) Xg'{x){l — x) + ^g”{x){px{l — x))^ + ax + 6 > 0. 


Intuition and smooth fit. We can imagine that the optimal strategy, in 
either the strong or the weak formulation, is of the following form: do not 
observe if is small, declare the alarm if vr^ is close to 1 and observe 
otherwise. More precisely, we postulate that there are two constants 0 < 
A < B < 1 such that on [0, A], it is optimal not to observe, on ]A,i?[ it is 
optimal to observe without declaring an alarm and on [il,l], it is optimal 
to stop and declare the alarm. That is, 

(4.4) and r* = inf{t > 0: vrf > B}. 

In order to satisfy condition (b) of Lemma 3.7, we need 

(4.5) Xg'{x){l — x) + ax = 0, xG]0,A] 

and 


(4.6) Xg'{x){l — x) + ax + ^g"{x)p‘^x'^{l — x)'^ + b = 0, x€]A,B[. 

In order to satisfy condition (d) of Lemma 3.7, we need 

(4.7) 5r(x) = l-x, xG[B,1]- 

In order to find an expression for g, it is natural to solve first the differ¬ 
ential equations (4.5) and (4.6) separately, that is, to seek two functions gi 
and g 2 such that 


(4.8) 


Xg[{x){l — x) + ax = 0, 


0 < X < A 
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and 

(4.9) \g' 2 {x){l — x) + ax+ ^g 2 {x)p^x^{l — x)"^+ b = Q, A<x<B. 

Three constants of integration will appear, one for gi and two for g 2 . These 
constants can then be determined by “pasting together” gi and g 2 , that is, 
requiring equalities such as 

(4.10) gi{A)=g2{A) 
and, by (4.7), 

(4.11) g 2 {B) = l-B. 

These two equalities are referred to as “continuous fit” [19]. As in most 
problems of optimal stopping or control, they are not sufficient to determine 
the five unknown constants, namely, the three constants of integration and 
the two “free boundaries” A and B. For this, it is necessary to use a version 
of the “principle of smooth fit”; see [19]. In particular, one can postulate 
that 

(4.12) g'2{B) = -1 
and 

(4.13) g[{A)=g' 2 {A). 

We need one more equation in addition to (4.10)-(4.13), since there are five 
unknown constants. Since we want to apply Ito’s formula, it is natural to 
want g to be twice differentiable at A. This gives one more equation, 

(4.14) g'l{A)=g'i{A). 

Solving the equations. We seek functions oi and go defined on [0,1] sat¬ 
isfying (4.8)-(4.14). Set 

fi{x) = g'l (x), f 2 {x) = g' 2 {x). 


The value of A. For 0 < x < A, differentiate (4.8) to get 
-A/i(x) A/((x)(l - x) -h a = 0, 


that is, 
(4.15) 


fiix) 


Xfijx) - a 
A(1 — x) 


From (4.9), we get 


(4.16) 


—ax — b — A/ 2 (x )(1 — x) 
(l/2)p2x^(l — x)^ 
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By (4.14), if we plug x = A into (4.15), (4.16), we get 

_A(aA + b)- A2/2(^)(1 - Al) = -A) + ^A\l - A)MA). 

Since /2(^) = fi{A) by (4.13), we solve for fi{A), 


(4.17) 


(apV2)A^(l -A)- X{aA + b) 
^ (l-Al)(A2 + (Ap2/2)Al2) 


Plugging (4.17) into (4.8) gives an equation for A, whose solution is 


(4.18) 



For the observation region ]A, B[ to be nonempty, we must have A<1, 
but further, since we want gi to be concave by Lemma 2.8, we also must 
have 


(4.19) h{A) = g[{A) >-1. 
From (4.8), 

( 4 . 20 ) 


so (4.8) and (4.19) give —> —1, or equivalently, A < With (4.18), 
we conclude that the observation region ]A,B[ is not empty if 


(4.21) 


b< 


Xap^ 


2(a + A)2- 


Determining f 2 {x). For A< x < B, equation (4.9) becomes 
(4.22) A/2 (x)(1 — x) + ax + \f 2 {x)p^x^{l — x)^ + 6 = 0. 

A solution of the homogeneous equation 


>^f{x){l -x) + 4/'(x)/>^x^(l - x)^ = 0 


IS 

(4.23) 


f{x) = 


1 — X 


X 


„a/x 


where a = 


p2 • 


Therefore, the solution of the inhomogeneous equation (4.22) is 

(4.24) h{x) = Kifix) + fix) y 2 l{Vy )2 f\y) dy- 

From (4.13) and (4.8), we conclude that 
/. N o. A 1 

^ ^^~Xl-Af{Ay 

Formulas (4.25) and (4.24) together determine /2(x). 
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Remark 4.1. In the case where 6 = 0, then A = 0 by (4.18), and we 
must have Ki = 0 in order that f 2 {x) be bounded. This recovers the case 
discussed in [23], Chapter 4.4. Therefore, we consider the case 6 > 0. 

Determining B. Observe that 

lim/(x) = 0 and lim/2(3:) = —oo. 

fC —>-1 

Indeed, the first equality is obvious, and the second holds because for x 
near 1, 

fix) ~ (1 

and, using rHopital’s rule, 

f 2 ix) ~ ^(1 - x)“ ^ dy ~ -(1 - x)i+“(l - 

~ —(1 — 

Therefore, if (4.21) holds, then / 2 (^) = Kif{A) = —> —1, so there is 
B G ]A, 1[ such that 

(4.26) /2(i?) = -1. 

With this choice of R, (4.12) is satisfied. The next lemma shows that in fact, 
there is only one solution to (4.26). 


Lemma 4.2. The function /2 defined in (4-24) ‘is strictly decreasing on 
[^, 1[, and therefore, there is a unique B g]^,1[ satisfying (4-26). 


Proof. By (4.22), 
(4.27) f'fix) 

where 


2A 1 
x^(l — x) 


{il{x) - /2(x)), 


V’(x) 


ax + h 
A(1 — x) 


Therefore, f^ix) < 0 if and only if fi{x) < f 2 ix). In fact, we will see in (4.51) 
[see also (4.32)] that 

f 2 ix)>xe]A,l[. 

— x 


We conclude that f^ix) < 0 for x G]A, 1[, and this proves the lemma. □ 






A QUICKEST DETECTION PROBLEM 


19 


Remark 4.3. It is clear from (4.18) that the value of the constant A 
is a continuous function of the observation cost b. The same is true for the 
constant B, by the following argument. 

We make explicit the dependence of /2 on b by writing f 2 {x,b). Equa¬ 
tion (4.26) becomes f 2 {B,b) = —1. We see that ^{x,b) > 0 by differentiat¬ 
ing under the integral sign in (4.24). Therefore, the implicit function theorem 
implies that R is a continuous (and even differentiable) function of b. 


Determining g 2 {x). Because g 2 {x) = f 2 {x), g 2 {x) can be written 

(4.28) g2{x)= [ f2{y)dy + K2. 

Ja 


From (4.11), we see that 


(4.29) 
so that 

(4.30) 


K 2 = 1-B- r f 2 iy)dy, 
Ja 



f 2 {y)dy + l -B. 


Determining gi{x). Because gi{x) = /i(x), gi{x) can be written 

(4.31) g-^(x)= [ fi{y)dy + K3, 

Ja 

where fi{x) is determined from (4.8), 

(432) /■(-) = 

From (4.10), (4.28) and (4.31), we get 

(4.33) K3 = K2. 

We can perform the integration in (4.31) to get 

(4.34) gi{x) = ^( 3 : + ln(l -x) - A- ln(l - ^)) -L K 2 , 

A 

with K 2 determined by (4.29). 

We have now found two functions gi and 52 that solve (4.8)-(4.14). In 
order to ensure that this solves our optimal control problem, slightly more 
is needed: in particular, we need inequalities (4.2) and (4.3) for all x G [0,1]. 
Set 

(4.35) Lig{x) = Xg'{x){l — x) + ax, 

(4.36) L 2 g{x) = A 5 '(x)(l — x) + ax + \g"{x)p^x^{l — x)^ -|- b. 
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Proposition 4.4 (Candidate value function). Suppose that 0 < b < 
Aa/ 0 ^/( 2 (a + A)^). Define g{x) on [0,1] by 


(4.37) 


9{x) 


gi{x), ifO<x<A, 

g 2 {x), ifA<x<B, 

1 — X, if B <x <1, 


where A is defined in (4-18), and B is defined in (4-26). Then g is strictly 


eoncave in [0,5], 

and 

(4.38) 

0 < g{x) <1 — X, 0<x<l 

(4.39) 

Lig{x) =0, 0 < X < A, 

(4.40) 

L2g{x)=0, A<x<B. 

Furthermore, 


(4.41) 

L2g{x) >0, 0 < X < A, 

(4.42) 

Lig{x) >0, A < x < B, 

(4.43) 

Lig{x) >0, 5 < X < 1, 

(4.44) 

L2g{x) >0, 5 < X < 1. 


Proof. Properties (4.39) and (4.40) follow from the construction of gi 
and g 2 ] see (4.8) and (4.9). The strict concavity of gi and 52 (hence of g 
on [0,i?]) follow from (4.32) and Lemma 4.2. This concavity property and 
(4.26) imply g{x) <1 — x, 0<x<l. Finally, since g'fix) = /i(x) < 0 for 
0 < X < ^ and g' 2 {x) = / 2 (x) < 0 for ^ < x < S, 5 is nondecreasing on [0, B ], 
therefore nonnegative on [0,5] since g 2 {B) = 1 — B >0. This proves (4.38). 

Note that (4.43) implies (4.44), and on [5,1], (4.43) becomes — A(1 — x) + 
ax > 0, that is, x > Therefore, (4.43) will hold provided we show that 


(4.45) 


B> 


A 

u T A 


To see this, note from (4.32) that 


(4.46) 


fi 


u T A 


We shall show that 


a X /(u T A) 

A ■ l-A/(a + A) 


(4.47) 


f 2 {x)>fi{x) forx>^. 


Then, (4.47) and (4.46) imply that 

A 


/2 


a X 


> -1 


that is, B > 


X 

u T A 
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[since f 2 {x) < —1 for x > B, hy (4.26) and Lemma 4.2], proving (4.45). 

It remains to prove (4.47). Set h{x) = f 2 {x) — fi{x). From (4.8) and (4.9), 
we see that for x > A, 

(4.48) Ah(x)(l — x) + ^h'{x)p^x^{l — x)^ + 6 + ^ f[{x) x"^ {1 — x)'^ = 0. 

By (4.20), 

a 1 


(4.49) 

so (4.48) becomes 




A (1 — x)2 ’ 


(4.50) Xh{x){l — x) + -h'{x)p'^x‘^{l — x)^ + 6 — ^^x^ = 0. 

2 2A 

Recall from (4.18) that b — ^x^ < 0 for x > ^. We note that h{A) = 
h'{A) = 0 by (4.13) and (4.14), and from (4.50), the following holds: for 
X > A, it is not possible to have simultaneously h{x) < 0 and /i'(x) < 0. 
Since h{A) = 0, this implies that for x > A, h{x) cannot be negative (since 
otherwise, there would be y e]^,x[ with h{y) < 0 and h'{y) < 0), therefore 
h{x) > 0 for X > A, that is, 


(4.51) 


f 2 {x)>fi{x) for X > ^. 


This proves (4.47). Therefore, (4.43) is proved. 

To check (4.41), we use (4.8), to see that for 0 < x < A, 


L2g{x) = (x)p^x^(l - x)^ + 6, 


and from (4.49), 


therefore. 


5i'(x) = -^ 


a 1 


A (1 — x)2 ’ 


L 2 g{x) = -^p x^ + b, x<A, 

and the right-hand side is nonnegative for x < ^ by (4.18). This proves 
(4.41). 

Finally, (4.42) is a consequence of (4.47), since (4.47) implies that 


Lig2{x) > Ligi{x) = 0 . 


□ 
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Case where b > this case, we postulate that the observation 

region ]A,B[ is empty (i.e., B = A), so we seek gi{x) such that 


(4.52) A5Ux)(1- 

(4.53) 

(4.54) 

From (4.52), we see that 

(4.55) 9'i{x) = - 


x) + ax = 0, 0<x<B, 

gi{B) = l-B, 

ax a / 1 \ 

Al — X A\ 1 — xy 


so for some constant K to be determined, 

(4.56) gi{x) = K + ^x + ^ln{l — x). 

A A 

From (4.55) and (4.54), we see that 


that is, 

(4.57) 



1-B 


= -l, 


B = 


A 

u 4- A 


From (4.53) and (4.56), we obtain 


Therefore, 

(4.58) 


K = 1-B-^B-^ ln(l - B) 
A A 



—)■ 

a + Ay 




We note that ^((x) is decreasing, ^((0) = 0 and 9 'i{B) = — 1, so 1 — x > 5 i(x) 
for 0 < X < i?, by (4.53). Since 1 —x > a/{a + \) = 1 — B for x < B, gi{x) > 0 
for 0 < X < i?. 


Proposition 4.5 (Candidate value function). Suppose that b>Xap'^j 
(2(a + A)^). Define gi{x) as in (4-58) and g{x) on [0,1] by 


(4.59) 


\9i{x), ifO<x<B, 

\ 1 — X, if B < X <1, 


where B is defined in (4-57). Then g is strictly concave on [0, B], 


(4.60) 0 < g'(x) < 1 — X, 0 < X < 1, 

(4.61) Lig'(x) = 0, xG[0,B], 









and furthermore, 

(4.62) 

(4.63) 
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Lidix) > 0 for B <x <1, 
L 2 g{x) > 0 for 0 < X < 1. 


Proof. Property (4.61) follows from (4.52), and the strict concavity of 
gi, hence of g, on and (4.60) are established just after (4.58). 

Note that for B < x <1, 

Lig{x)>0 —A(l —x) + ax>0 


X > 


n T A 


= B, 


and this is indeed that case, so (4.62) holds. 

For B <x <1, L 2 g{x) = Lig{x) + b, and both of these terms are nonneg¬ 
ative, so L 2 g{x) > 0 for these x, proving part of (4.63). 

For 0 < X < S, 

L 2 g{x) = L 25 i(x) = Ligi{x) -L \g'l{x)x^{I - xf + b. 


Since Ligi(x) = 0, 

L2g{x) = 


la —1 


p^x^{l — xY + b = — 


2A(l-x)2 


ap^x"^ 

2A 


+ b, 


so 


L2gix) > 0 


ap^x"^ 

2A 


<h 


X < 


l 2Xb 

ap'^ 


This will hold for x < B provided it holds for x = B. Now 
B < 


>2Xb 


ap^ 


X 


a “b A 


“ ap^ 


b> 


Xap^ 


2(a + A)2’ 


which is the assumption of this case. This proves (4.63). □ 


Comments on the optimal strategy. In the case where b > Xap^/{2{a + 
A)2), the observation region is empty, the candidate optimal control is /i* = 0 
[with this control, (2.2) obviously has a strong solution] and the candidate 
optimal stopping time is 

(4.64) T* = mi{t>0:TT;>B}, 
where (vr*) is defined by 

(4.65) dvTj = A(1 — TTj) dt, ttq = ttq 

[so TTj = TT^*, where is defined in (3.6) with h there replaced by h*]. It 

is straightforward to check that {h*, t*) is indeed an optimal strategy (both 
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in the weak and strong formulations), and we do this in Section 5 in the 
proof of Theorem 5.1. 

On the other hand, in the case where b < Xap^/{2{a + A)^), the optimal 
strategy should take the form mentioned in (4.4), 

(4.66) h* = and r* = inf{t > 0: vr^ > i?}, 

where the law of (tt*) should be determined by the diffusion equation 

(4.67) dnl = A(1 - TT^) dt + dWt, 

or, looking back to (3.8) and (3.6), 

dn* = A(1 - 7r*)dt + ^<(1 - Ol{ 7 r*>A}(’’l{ 0 <t} dt + (^dWt) 

(4.68) 

- ^«)^(l-7rt*)l{^*>A} dt. 

Because of the irregularity of p i—)• l{p>A}) equations such as (4.67) and (4.68) 
do not have a strong solution in general (see, e.g., [8, 14, 24]), but according 
to the theory developed in [12], Chapter 5, Section 24, they do have a weak 
solution [such that the process (tt^) spends an amount of time at A that has 
positive Lebesgue measure]. Therefore, from the discussion in (3.8)-(3.15), 
we expect (4.66) to determine an optimal control system in the weak formu¬ 
lation of our problem, but there will be no optimal strategy in the strong 
formulation! This means that we will be able to use verihcation Lemma 3.7 
to prove, in Section 5, that the function g dehned in Proposition 4.4 is equal 
to the value function p", but a different approach via e-optimal strategies 
will be used to show that g is equal to g. 

5. The value function. Formulas (4.37) and (4.59) provide candidates, 
denoted by g, for the value functions g and g'" defined, respectively, in (2.5) 
and (2.6). The objective of this section is to prove that indeed, these two 
value functions are equal, and equal to g. 

Theorem 5.1. (a) Case where 0 <b < \ap^/{2{a -|- A)^). Define A by 

(4-18), let f be as in (4-23), Ki as in (4-25), /2 as in (4-24), B as in 
(4-26), K 2 as in (4-29), gi as in (4-34) o,nd 52 as in (4-30). Then the 
function g defined in (4-37) is equal to the value function g^ defined in 
(2.6). Further, the control system associated to h{t,p) = l{p>A} and to t* in 
(4-66) is optimal. 

(b) Case where 6 > Aa/9^/(2(o-|-A)^). Define B by (4-57) and gi by (4-58). 
Then the function g defined in (4-59) is equal to the value function g'" defined 
in (2.6). 
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Theorem 5.2. In both cases of Theorem 5.1, the two value functions 
g (strong formulation) and g'^ (weak formulation), defined, respectively, in 
(2.5) and (2.6), are equal (and equal to the function g of Theorem 5.1). 

Remark 5.3. It is interesting to observe how the value function g and 
the thresholds A and B depend on the observation cost b: we write g{x,b), 
A{b) and B{b) to indicate this dependence. 

From (2.4) and (2.5), be^g{x,b) is nondecreasing. For 6 = 0, g{-,0) is 
the value function obtained in [23], Chapter 4.4, Theorem 9. As 6 increases 
from 0 to 6 c = Ao/ 9 ^/( 2 (a + 6 )^), B{b) decreases from R(0) to B{bc), and A{b) 
increases from 0 to A( 6 c) = A/(a + A) = B{bc) [see (4.18) for the first equality 
and the second follows from the lines preceding (4.26) since / 2 (A( 6 c)) = —Ij. 
For 6 > 6c, g{-,b) = g{-,bc) since there is no dependence on 6 . 

Theorem 5.1 will be proved in two steps. We begin by showing that g < g^. 

Lemma 5.4. In both cases (a) and (b) of Theorem 5.1, the inequality 
g < g'^ holds. 


Proof. We are going to use part (1) of Lemma 3.7. Suppose first that 
we are in case (a) of Theorem 5.1. By construction, and in particular by 
(4.10), (4.13) and (4.14), g is on [0,R[, and on [0,1] by (4.12) and 
(4.7), so 

(5.1) g'^B-)=g\B+) = -l. 


By (4.38), 0 < g{x) <l — x. Let {{ht),T,X) be a control system, and set 

(5.2) Yt = g{TTi) + a f Tr^ds + b f hgds. 

Jo Jo 

We now apply Ito’s formula, in the form given in [19], Section 3.5: 

Yt = Yo+ [ g’in')) dTT^ + f {mr’) + bh^) ds + ^ f g"( tt^) d{Tr^)^ 

Jo Jo ^ Jo 


(5.3) 


+ 2 ^ 9 '{B+) - g'{B-))Lf , 


where is the local time of (tt^) at B. By (5.1), the factor g'{B+) — g'{B—) 
vanishes, so as in (4.1), we find that 

(5.4) Yt = Yo+ rg'{7:’f)-7r’f{l-7:^)^sdWs+ T $(7r^ h,) ds, 

Jo ^ Jo 

where 


<^{x, g) = Lig{x) + g[y [x){px{l - x)f + 6] 


rye [0,1], 
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and Li is defined in (4.35). We note that by construction and by Proposi¬ 
tion 4.4, 

<h(3:, 0) = Li 5 r(x) > 0 for all X G [0,1], 

^(x, 1) = L 2 g{x) > 0 for all x G [0,1] \ {i?}, 

where L 2 is defined in (4.36), and since rj 1 —)• ^(x,r]) is an affine function, we 
conclude that ^{x,r]) > 0, for all rj G [0,1]. Since g' is bounded on [0,1], the 
stochastic integral in (5.4) is an J^^-martingale [recall (3.5)], and therefore 
{Yt) is an J^^-submartingale. The conclusion now follows from part (1) of 
Lemma 3.7. 

Now suppose that we are in case (b) of Theorem 5.1. By construction, g 
is on [0,B[, and on [0,1] by (4.53) and (4.54), so 

g'{B-)=g'iB+) = -l. 

By (4.60), 0 < g{x) < 1 — x, for all x G [0,1]. Let {{ht),T,X) be a control 
system, and dehne Yt as in (5.2). Applying Ito’s formula, we obtain (5.3), 
and this leads again to (5.4). Using this time Proposition 4.5, we see that 
‘h(x,? 7 ) > 0, for all g G [0,1]. Therefore, we conclude, as before, that (Yt) is an 
J-'^-submartingale, and the conclusion follows from part (1) of Lemma 3.7. 
□ 


We now prove Theorem 5.1. 

Proof of Theorem 5.1. We begin with case (b). As mentioned in 
(4.64) and (4.65), the candidate optimal control system is {h*, t* ,X*), where 
/i* = 0, X* = 0 and 

T* = inf{t > 0: TT* > B}, 

where (ttj) is dehned in (4.65). Clearly, (h*, t* ,X*) is a control system, and 
so it suffices to check properties (b), (c) and (d) of Lemma 3.7. By (4.61), 

dY* = Lig^TTt) dt = 0 fort<r*. 

Therefore, (T)])^^*) is a (constant and deterministic) martingale, proving (b). 
Further, since (tt^) is deterministic, we solve (4.65) to find that 

(5.5) 

I 0, if VTO > B, 

so (c) holds. Finally, if ttq < B, then 

g{7r%)=g{B) = l-B = l-7r;, 
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by (4.53), and if ttq > B, then 


9«*) = ai'^o) = 1 - vr^ = 1 - < 


by (4.59). This proves case (b) of Theorem 5.1. 

We now consider case (a). We have seen in Lemma 5.4 that g <g^■ In 
order to establish the converse inequality, consider (/i*) and t* defined in 
(4.66) and the associated control system {{h*),T* ,X*), constructed as in 
(3.8)-(3.15), using the function h{t,p) = l{p>A} and n* defined as a weak 
solution of (4.68). Then for t <t* , 


\Lig{TTl), if7r|<^ 

XL^gint), ii7rte[A,B[ 


= 0 


by (4.39) and (4.40). Therefore, is an ^-'"’‘"-martingale. According to 

Lemma 5.5 below, < oo, and g{TT^l) = g{B) = 1 — B hy (4.66) and 

(4.37). This proves properties (b), (c) and (d) of Lemma 3.7 and concludes 
the proof that g = g^ and {{h*),T* ,X*) is an optimal control system, since 
we already verified (a) of Lemma 3.7 during the proof of Lemma 5.4. □ 


Lemma 5.5. Suppose that we are in case (a) of Theorem 5.1. Let t* be 
defined as in (4-66). Then for all vr G [0,1], £'^(r*) < oo. 

Proof. If vr G [B, 1], then Et^{t*) = 0, and if vr G [0, A[, then (vr*) reaches 
A at the deterministic time ln((l — vr) /(I — A)) [see (5.5)], so the problem 
reduces to considering vr G [A,B[. 

Recall from (3.14) and (4.67) that (vr^) solves, in the terminology of [12], 
Chapter 5, Section 24, an s.d.e. with delayed reflection at the boundary 
point A, and this process is associated to a diffusion with instantaneous 
reflection at the boundary 

(5.6) = A(1 - 6) dt + pit{l - it) dWt + dCt, 

where (Ct) is a nondecreasing process that increases at those points where 
it = A, 

and (ft is dehned by the relation 

r<l>t 

As explained in [12], (vr^) has the same law as (^rt); where Tt is defined by 
the relation 


1 
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Therefore, t* has the same law as T = inf{t € M_|_ = B}. Letting a = 

inf{s G M+ = -B}; we see that a = tt- Further, according to Lemma 5 in 
[12], Chapter 5, Section 23, ET^{a) = Vb(y) — Vb('/r), where Vq{A) = 0 and for 

yG]Ai[, 


A(i - y)Fo'(2/) + ipV(i - y )X"(y) = 1- 


An explicit expression for Vq can be obtained by using (4.24) with Ki = 0, 
a = 0 and 6 = — 1, and then integrating from A to y. In particular, < 

+ 00 . 

Notice that 



Therefore, it suffices to show that < +oo. By (5.6), 


rtAcr rtAcr 

(5-7) ItAo- = fo + / A^~is)ds+ / pisi^ — is) dWs + CtAa- 
Jo Jo 


The stochastic integral is an L^-bounded martingale, since 



Therefore, the optional sampling theorem can be applied and, since the ds- 
integral in (5.7) is nonnegative, we find that 


B — F^7r(^(T) > vr + 


so ill^(Co-) < + 00 , as was to be proved. □ 

For the remainder of this section, we put ourselves in case (a) of Theo¬ 
rem 5.1. Since we have observed just after (2.6) that y > y", and = y by 
Theorem 5.1(a), in order to prove Theorem 5.2, it suffices to establish the 
inequality y > y. For e > 0, we are going to define an admissible control h^, 
and a strategy (/i'^,t^), with associated cost y^ = £'(C'(/i^,r^)), and we shall 
show that ye —)• y as e j, 0. From the definition of y in (2.5), this will establish 
that y > y, and this will prove Theorem 5.2. 

An almost optimal strategy. Define the function 



Consider the s.d.e. 


dpl = K'^-pl)dt 


(5.8) 



r 


,2 


2 




a- 
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with Pq = ttq. According to [13], Theorem 3.2 page 168, this s.d.e. has a 
unique strong solution {pf,t > 0), since is Holder-continuous with ex¬ 

ponent 1 / 2 . 

Set 

(mi{t>0:pl>B}, if {• • •} 7 ^ 0 , 

\ -|-oo, otherwise. 


(5.9) 


r" = 




Using (3.8)-(3.15), we associate to a strategy {{h‘ 

We are now going to determine the cost of the strategy and we 

will see in Proposition 5.7 below that for e small, this strategy is nearly 
optimal. Let 

(5.10) ge{TTo) = E{C{h^,T‘')). 

In order to determine the function g^, we will use the following lemma. 

Lemma 5.6. Suppose that we are in case (a) of Theorem 5.1 and that 
we can find a continuous function g^ on [0,1] that is on [0,1] \ {A,B}, 
on [0,1] \ {B} and such that 

(5.11) Lg^{x) =—{ax + bh^^\x)), 
where Lgi.{x) is defined by 

(5.12) Lge{x) = A(1 - x)g'fix) + |p^x^(l - xfh’^‘'\x)g''{x) 
and 

(5.13) geix) = l-x forxe[B,l]- 

If, in addition, 

(5.14) E,fiTe) < +00 
then ge = ge- 


for all X G [0,1], 


Proof. Suppose ttq G [B, 1]. Then ge{T^o) = 1 — tto, and since = 0 a.s., 
(2.4) gives 

5.(vro) = E{C{hfi T^)) = P{e>0} = l- vro. 

Therefore, by (5.13), ^^(Tro) = gei'^o) in this case. 

Now suppose that ttq G [0,H[. According to Lemma 2.9 and (3.13), 

ge{Po) = E{C{hfiP))=E(^l-p^^e+aJ^ p^ds + bj^ h''^\pl)ds^. 

Since P < -|-oo a.s. by (5.14), p%e = B, and 1 — B = ge{B) by (5.13), so 

E{C{hfiP)) = E(^e{p%e) + a plds + bj^ h^‘'\pl)ds'^. 
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As in Lemmas 3.5 and 3.6, we see from (5.8) and (3.9) that 


(5.15) dpf = A(1 - pf) dt + ppf{l - p^^)^Jh(^){p|)dWl, 


where (VLf) is an Brownian motion. Let 



We apply Ito’s formula in the form given in [19], Section 3.5.3, using the 
fact that g'^{A—) = g'^{A+), to see that 



+ [y'{pl)p\pl)\l-plfh^^\pl) + apt + bh%pt)] dt 



By (5.11), the drift in brackets vanishes, and therefore {Mt/\T^,t > 0) is an 
-martingale. Since, by (3.13), 0 < < 1, and 0 < < 1 and g^ is 

bounded, we see that \Mt\ < A -|- (a -|- h)t, so \Mt/\T^ | < A -|- (a -|- 6)r^. Since 
< -|-oo by (5.14), is uniformly integrable, and so 

geiTTo) = E{C{h^ ,T^)) = = ge{po) = ge{TTo). 

Therefore, ge(vro) = gei'^o) as claimed. This completes the proof of Lemma 5.6. 

□ 

Constructing g^. It remains to construct the function g^ satisfying the 
assumptions of Lemma 5.6. Notice that on ]0,A[, writing gi instead of g^, 
equation (5.11) becomes 

(5.16) A(1 — x)^! (x)-|-ax = 0, 

and as in (4.56), the solution of this differential equation is 


gi{x) = ^(x-hln(l -x)) -h ATf, 


(5.17) 


where is a constant to be determined. 

On ]A + e,B[, writing g^ instead of g^, equation (5.11) becomes 


(5.18) A(1 — x)g'^{x) + y^x‘^{l — x)‘^g'l{x) -|- ax -|- 6 = 0 


which is the same equation as in (4.9), and as in (4.28), its solution is 


(5.19) 



h3{y)dy + KI, 
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where 

( 5 . 20 ) h,i.) = Kim + /(.) 

and 

(5.21) f{x )=^—where a = ^, 

and iCf) .h'l are constants to be determined. 

Finally, on ]A,A + e[, writing 52 instead of ge, equation (5.11) becomes 

(5.22) A(1 — x)g 2 {x) H —— x)"^- - + ax + b- -= 0. 

2 (S S 

Let h 2 {x) =^ 2(^)5 associated homogeneous equation is 

(5.23) A(1 - x)f2ix) + ^P^x^{l - xf ^ ^ ^ Kix) = 0, 
whose solution is 


(5.24) 

where 


and 


fl{x)='ljje{x){x-A) 


(3e = 


1 2A£ 

A^{1 — A) 


V^£(x) = exp(^- 


2Ae 1 


Ap^ X 


Therefore, 

h2{x) = Kfl{x) 

(5.25) 

+ /K^^) 

and if we want /12 to be bounded as x A, then we must set iC = 0 (notice 
that there is no integrability problem at y = A). We conclude that 


-2e 


ay + b 


y-A 


e y y^(i-2/)^(y--4)/|(y) 


dy, 


(5.26) h2{x) 

and 

(5.27) 


flix) f 

Ja 


- 2 £ / 


p- V 


[ay + b 


y-A 


£ J y'^il-y^iy- A) fl{y) 


dy 


px 

g2{x)= h2{y)dy + Kl, 
Ja 


where iC| is a constant to be determined. 
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In order to determine the four constants itTf,..., we shall impose the 
four equations 

(5.28) UB) = l-B, 

(5.29) g 2 ,{A + e) = g 2 {A + e), 

(5.30) g''j^{A + e) = g'2{A + e), 

(5.31) gi{A) = g2{A). 

We note that (5.29) and (5.30), together with (5.18) and (5.22), imply that 
g'^{A + e) = g 2 {A + e), and (5.31), together with (5.16) and (5.22), implies 
that g[{A) =g' 2 {A). 

From (5.28) and (5.19), we see that 

(5.32) Kl = l-B-[ h3{y)dy, 

JA+e 

while (5.29), (5.19) and (5.27) imply that 

rA-\-£ ^ 

(5.33) Kl= h2{y)dy + Kl 


Equality (5.30), (5.20) and (5.26) give the relation 

mA + e) 


(5.34) 


f{A + s) 

-2e f 

"A 


ay + b 


y-A 


e ) y‘^{l-yY{y- A) fl^{y) 


dy, 


while (5.31), (5.17) and (5.27) give 


(5.35) 


A^ + i^^l_A)) + Kf = Kl 

A 


Therefore, (5.34) determines K^, (5.32) determines K^, then (5.33) deter¬ 
mines and (5.35) determines Kf. 


Proposition 5.7. Fors>0, let be determined by (5.32)- 

(5.35), define gi{x) as in (5.17), g 2 {x) as in (5.27), and g 3 {x) as in (5.19). 
Set 



if 0 < X < A, 

92{x), 

if A<x<A + e 

93{x), 

if A + e<x<B 

, 1 — X, 

if B <x <1. 


Then g^ satisfies the assumptions of Lemma 5.6. Further, let g he as in case 
(a) of Theorem 5.1. Then 


\\m.gi.{x) = g{x) /or aZZ x G [0,1]. 
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Proof. By the comments that follow (5.31), Qe is on [0,1] \ {A, B}, 
on [0,and continuous on [0,1]. For x G [B, 1], g£{x) = l — x = g{x), 
so we consider the case where x G [0,i?[. 

Case 1: x €]A,B[. We first check that iCf —)• Ki, where Ki is defined in 
(4.25). We note that 


KI 


V^£M_+£) -/3e 
fiA + e) 


X 



(^ay + b 




1 1 

J/2(l-y)2^^(y) 


{y-A)K ^dy. 


Notice that ijjeiA + e) —)• 1 and f{A + e) —)• f{A) as e 0. Set 


Aq — 


1 2A 


A^(l-A) p2 


so that Pe = AqS. 


Then 


KI 


1 


-1—Aoe 


f A.-\-£ 


f{A) 

1 -2 


Ia P" 
1 


-^{ay + b 


y-A 


y‘^{l - yfipeiy) 


(y_yl)Aoe-l^y 


f{A) A^il-Afi^M) 

A-\-£ 


X £ 


1-Aoe 


aA{y - + -[y - A)^o£ 


dy, 


and the integral is equal to 


and therefore, 


\miKl 

sr^l'O 


p\o£ 

aA— 

Aoe 


b 

£ Xo£ + 1 ’ 


1 —2 1 aA a A 1 

JiA)^ A^{l-Ayj;;^~Xl-AfiA) 


Ku 


as claimed. 

This implies that for y > A + £, h'i{y) -A f 2 {y), where and /2 are, 
respectively, defined in (5.20) and (4.24). By dominated convergence, we 
deduce that -A K 2 , and for x G ]A,B[ and for e 0 with 0 < £ < x — A, 


Peix) = gsix) -A g 2 {x) = g{x), 


where 52 is defined in (4.30). 

Case 2: x G [0, A] . From (5.33), we see that — iC| —)■ 0, therefore -A 

K 2 by the above, and from (5.35), we see that 

Kf^K2-^{A + ln{l-A)). 

A 
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We conclude from (5.17) and (4.34) that for x G [0,^], as e^O, 

9e{x) = 51 (a;) gi{x) = g{x). 

This completes the proof of Proposition 5.7. □ 

The next lemma checks condition (5.14). 

Lemma 5.8. Fix e > 0, and let be defined in (5.9). Then for all x G 
[ 0 , 1 ], £'a;(r^) < oo. 

Proof. We first seek a bounded function 7 ^ defined on [0,i?] such that 

(5.36) = 

where L is the operator defined in (5.12). 

For 0 < X < A, (5.36) becomes 

(5.37) A(1-a:) 7 '(x) =-1, 
so 

(5.38) 7 e(x) = — ln(l — x) + T>i, {)<x<A. 

A 

For A<x < A + £, (5.36) becomes 

(5.39) A(1 - x)-i'fix) + \p^x‘^{l - x) 2 ^^ 7 ''(x) = -1, 
and as in (5.22) and (5.25), the solution to this equation is 

(5.40) ^fix)= [ hi{y)dy + D'i, A<x<A + £, 

Ja 

where 

(5.41) h4^{x) = D2flix) + fl{x) [ L 

Ja r y^A-yr{y-^) fliv) 

and /I is defined in (5.24). Since we want /14 and 7 ^ to be bounded (as 
X fA), we set D 2 = 0. 

For A + £ < X < B, (5.36) becomes 

(5.42) A(1 - x) 7 '(x) + 5 pV(l - x)^ 7 "(x) = -1, 
and as in (5.19), the solution of this equation is 

(5.43) 7 g(x)=/' h5{y)dy + D4, 

JA+e 
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where 

(5.44) /.,(x) = 0,/(x) + /(X) <‘y. 

and f{x) is defined in (5.21). 

We must determine the constants Di,..., D^. For this, we impose the 
following conditions: 


(a) 7 £(.B) = 0, 

(b) 7e((x4 + e)+) = 7e((x4 + e)—), 

(c) 7'((x4 + e)+) = 7'((x4 + e)-), 

(d) 7 £(x 4+) =7e;(x4-). 

We note that (b) and (c), together with (5.39) and (5.42), imply that 

(5.45) ^''((7l + e)+) = ^''((7l + e)_), 

so 7 £ will be at A + s. Also, (d) together with (5.37) and (5.39) implies 
that 

(5.46) 7'(A+) =7'(A-), 
so 7 e will be at A. 

From property (c), (5.44) and (5.41), we see that 

rA+e _ 2 £ I 1 


{A + e) — /|(A + e) 


'a y^(i-y)^(y-^)/|(y) 


dy, 


and this determines (and therefore / 15 ). 

From (a) and (5.43), we hnd that 

rA-\-£ 

D4= h5{y)dy, 

JB 

SO that 

(5.47) j^{x)= [ h^{y)dy for A + ■ 

JB 

From (b), (5.47) and (5.40), we see that 

rA+e rA+e 

/ h 5 {y)dy= / h 4 (?/) dy + ZI 3 , 

Jb Ja 

and this determines ZI 3 . 

Finally, from (d), (5.38) and (5.40), we see that 


£ <x < B. 


-ln(l-A) + F)i =D 3 , 


and this now determines Di. 
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With the choice of constants Di,... ,D^ above, we have determined a 
function 7 ^ : [0,5] —)• M which is on [0,B] and on [0,A] and [^,-B]. 

We now turn to the study of For x G [A,B], the behavior of pf 

while pf G [A, ^ + e[ is somewhat unusual, because of the square-root in the 
diffusion coefficient in the s.d.e. ( 5 . 15 ). We are going to check below that in 
this interval, p^ — A\s comparable to the time-change (under a well-behaved 
time change) of a BESQ-process [ 22 ], Chapter XI, so it behaves essentially 
like the Cox-Ingersoll-Ross process [ 16 ], Theorem 6 . 2 . 3 , Proposition 6 . 2 . 4 . 
In particular, pl> A since x > ^, so (pf) never goes strictly below A (though 
it may hit A and A is instantaneously reflecting), and A -|- e is hit in finite 
time because pf is either recurrent or transient, depending on the values of A 
and p. On the other hand, h^^\pt) = 1 while pf G {A + £,B\, so ( 5 . 15 ) simply 
describes there a diffusion with positive, bounded and Lipschitz continuous 
drift and diffusion coefficients. 

Regarding the behavior of pf while G [A,A + e[, set pf = Pt ~ By 
( 5 . 15 ), 

( 5 . 48 ) dpl = \{l-A-pl)dt + ^/^atdWl, 

where 

at = p£~^/‘^{pl + ^)(1 - ^ - Pf)- 


In particular, there are two positive and finite constants and Cg such that 
Ce<at< Cs as long as pf ^[A,A + e[. Define a martingale 


(5.49) 
so that 


Mt= fasdW^, 
Jo 


= / aids. 


{M)t = [ 
Jo 


Define the increasing process {pt) so that {M)p^ = t, and notice that 


t 


t 


cjt < {M)t < C^t and < pt < 

Be Ce 

By (5.48) and (5.49), 

Pt=Po + [ K'^-A-pl)ds+ f y/^^dMs. 

Jo Jo 

Using the time-change formulas for deterministic and stochastic integrals 
(see Problem 4.5 and Proposition 4.8 in [15], Chapter 3, we see that 


Pt=Po 


r{M)t 1 AMP 

+ Xil-A-p%)—ds+ V^sdW!. 
Jo Jo 
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Setting Qs = Pp ^, so that pf = q(M)t > we find that 

f{M)t I r{M}t 

(l{M)t= X{ 1 -A-qs)—ds+ y/^dW^, 

Jo Jo 

and, setting t = pu, 


pu pu 

qu= X{1- A- qs)—ds+ / y/^dW' 

Jo ^Ps Jo 


The drift of {qu) is X{1 — A — qu)/> A(1 — ^4 —e)/C'|, so by the comparison 
theorem for s.d.e.’s [15], Chapter 5, Proposition 2.18, qu is greater than the 
BESQ-process with drift A(1 — A — £)jC‘^, hence qu>0 a.s., or, equivalently, 
Pt>A a.s. 

We now apply Ito’s formula to since 7 ^ is on [A,B\, 

/ tAr^ 1 ptAr^ 

l'e{pl)dpl + ^ ie{pl)d{p^)s 

ptAT^ ! - 

= le{pl) + 


+ 


ptAr^ 

/ L-ysipDds. 
Jo 


According to (5.36), L'y^{pl) = —1 for s < so, taking expectations, we 
find that 


-Ex(7e(P?ArO) = leix) - E^t A A), 


SO 

EAt A A) = -E:, (7£ (pIat^)) + le{x). 

The right-hand side is bounded, so sup^g^^ E 3 ;(t A r^) < -|-oo. By the 
monotone convergence theorem, Ex{t^) < -boo as claimed [and in fact, 
Ex{t^) = le{x)], X e [A,B]. 

For X G [0, A[, we observe from (5.15) that pf is deterministic and increases 
at speed > A(1 — A) until reaching A. Thus A is hit in less than some tq 
units of time, and so 

Ex{A) < To -b Ea{A) < -boo. 

Finally, for x G [B, 1], = 0 Px-a.s., so Ex{P) = 0. This proves Lemma 5.8. 

□ 


Lemma 5.9. The function defined in Proposition 5.7 is the cost as¬ 
sociated with the strategy {h^,T^), that is, for all x G [0,1], ge{x) = gfix) = 
E{C{h^,T^)) is defined in (5.10)]. 
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Proof. According to Proposition 5.7, satisfies the assumptions of 
Lemma 5.6, and according to Lemma 5.8, (5.14) holds. Therefore, by Lem¬ 
ma 5.6, = gei and this proves Lemma 5.9. □ 

Proof of Theorem 5.2. In case (a) of Theorem 5.1, in view of the 
considerations that follow the proof of Theorem 5.1, it remains only to prove 
that g < g. By definition of g and Lemma 5.9, the inequality g < g^ holds. 
Since g = limg^o de by Proposition 5.7, we conclude that g < g. This com¬ 
pletes the proof of Theorem 5.2 in case (a) of Theorem 5.1. 

The statement of Theorem 5.2 in case (b) of Theorem 5.1 follows from the 
fact that the optimal control system exhibited in the proof of Theorem 5.1 
is (trivially) a strategy, which then is necessarily optimal. □ 
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